Research on real-time physics-based deformation for haptic-enabled medical simulation

This study developed a multiple effective visuo-haptic surgical engine to handle a variety of surgical manipulations in real-time. Soft tissue models are based on biomechanical experiment and continuum mechanics for greater accuracy. Such models will increase the realism of future training systems and the VR/AR/MR implementations for the operating room

Augmented reality-based visual-haptic modeling for thoracoscopic surgery training systems

Background: Compared with traditional thoracotomy, video-assisted thoracoscopic surgery (VATS) has less minor trauma, faster recovery, higher patient compliance, but higher requirements for surgeons. Virtual surgery training simulation systems are important and have been widely used in Europe and America. Augmented reality (AR) in surgical training simulation systems significantly improve the training effect of virtual surgical training, although AR technology is still in its initial stage. Mixed reality has gained increased attention in technology-driven modern medicine but has yet to be used in everyday practice. Methods: This study proposed an immersive AR lobectomy within a thoracoscope surgery training system, using visual and haptic modeling to study the potential benefits of this critical technology. The content included immersive AR visual rendering, based on the cluster-based extended position-based dynamics algorithm of soft tissue physical modeling. Furthermore, we designed an AR haptic rendering systems, whose model architecture consisted of multi-touch interaction points, including kinesthetic and pressure-sensitive points. Finally, based on the above theoretical research, we developed an AR interactive VATS surgical training platform. Results: Twenty-four volunteers were recruited from the First People's Hospital of Yunnan Province to evaluate the VATS training system. Face, content, and construct validation methods were used to assess the tactile sense, visual sense, scene authenticity, and simulator performance. Conclusions: The results of our construction validation demonstrate that the simulator is useful in improving novice and surgical skills that can be retained after a certain period of time. The video-assisted thoracoscopic system based on AR developed in this study is effective and can be used as a training device to assist in the development of thoracoscopic skills for novices

Trustworthy and Intelligent COVID-19 Diagnostic IoMT through XR and Deep-Learning-Based Clinic Data Access

This article presents a novel extended reality (XR) and deep-learning-based Internet-of-Medical-Things (IoMT) solution for the COVID-19 telemedicine diagnostic, which systematically combines virtual reality/augmented reality (AR) remote surgical plan/rehearse hardware, customized 5G cloud computing and deep learning algorithms to provide real-time COVID-19 treatment scheme clues. Compared to existing perception therapy techniques, our new technique can significantly improve performance and security. The system collected 25 clinic data from the 347 positive and 2270 negative COVID-19 patients in the Red Zone by 5G transmission. After that, a novel auxiliary classifier generative adversarial network-based intelligent prediction algorithm is conducted to train the new COVID-19 prediction model. Furthermore, The Copycat network is employed for the model stealing and attack for the IoMT to improve the security performance. To simplify the user interface and achieve an excellent user experience, we combined the Red Zone's guiding images with the Green Zone's view through the AR navigate clue by using 5G. The XR surgical plan/rehearse framework is designed, including all COVID-19 surgical requisite details that were developed with a real-time response guaranteed. The accuracy, recall, F1-score, and area under the ROC curve (AUC) area of our new IoMT were 0.92, 0.98, 0.95, and 0.98, respectively, which outperforms the existing perception techniques with significantly higher accuracy performance. The model stealing also has excellent performance, with the AUC area of 0.90 in Copycat slightly lower than the original model. This study suggests a new framework in the COVID-19 diagnostic integration and opens the new research about the integration of XR and deep learning for IoMT implementation

Digital Twin-enabled IoMT System for Surgical Simulation using rAC-GAN

A digital twin-enabled Internet of Medical Things (IoMT) system for telemedical simulation is developed, systematically integrated with mixed reality (MR), 5G cloud computing, and a generative adversarial network (GAN) to achieve remote lung cancer implementation. Patient-specific data from 90 lung cancer with pulmonary embolism (PE)-positive patients, with 1372 lung cancer control groups, were gathered from Qujing and Dehong, and then transmitted and preprocessed using 5G. A novel robust auxiliary classifier generative adversarial network (rAC-GAN)-based intelligent network is employed to facilitate lung cancer with the PE prediction model. To improve the accuracy and immersion during remote surgical implementation, a real-time operating room perspective from the perception layer with a surgical navigation image is projected to the surgeon’s helmet in the application layer using the digital twin-based MR guide clue with 5G. The accuracies of the area under the curve (AUC) of our new intelligent IoMT system were 0.92, and 0.93. Furthermore, the pathogenic features learned from our rAC-GAN model are highly consistent with the statistical epidemiological results. The proposed intelligent IoMT system generates significant performance improvement to process substantial clinical data at cloud centers and shows a novel framework for remote medical data transfer and deep learning analytics for digital twin-based surgical implementation

A Comparison of Hour-Ahead Solar Irradiance Forecasting Models Based on LSTM Network

The intermittence and fluctuation character of solar irradiance places severe limitations on most of its applications. The precise forecast of solar irradiance is the critical factor in predicting the output power of a photovoltaic power generation system. In the present study, Model I-A and Model II-B based on traditional long short-term memory (LSTM) are discussed, and the effects of different parameters are investigated; meanwhile, Model II-AC, Model II-AD, Model II-BC, and Model II-BD based on a novel LSTM-MLP structure with two-branch input are proposed for hour-ahead solar irradiance prediction. Different lagging time parameters and different main input and auxiliary input parameters have been discussed and analyzed. The proposed method is verified on real data over 5 years. The experimental results demonstrate that Model II-BD shows the best performance because it considers the weather information of the next moment, the root mean square error (RMSE) is 62.1618 W/m2, the normalized root mean square error (nRMSE) is 32.2702%, and the forecast skill (FS) is 0.4477. The proposed algorithm is 19.19% more accurate than the backpropagation neural network (BPNN) in terms of RMSE

Machine Learning-Based Stealing Attack of the Temperature Monitoring System for the Energy Internet of Things

With the development of the Energy Internet of Things (EIoT), it is of great practical significance to study the security strategy and intelligent control system for solar thermal utilization system to optimize the operation efficiency and carry out intelligent dynamic adjustment. For buildings integrated with solar water heating systems, computational fluid dynamics simulation was used in analyzing the process of solar energy output. A method based on machine learning is proposed to predict energy conversion. Besides, the simulation and analysis are carried out in combination with the possible safety problems such as the vibration of the control system. This paper proposed a novel platform of EIoT for machine learning-based cybersecurity study and implemented the platform for the temperature monitoring system. After the evaluation of the machine learning-based cybersecurity study, the EIoT system demonstrated a high performance with the Extreme Gradient Boosting (XGBoost) training algorithm

Dichoptic Color Difference Threshold Decreases as Brightness Increases

Lustre perception is an important visual ability, but estimating the strength of perceived lustre is a computational challenge for both human and machine vision. We studied the characteristics of binocular lustre perception using the Dichoptic Color Difference Threshold (DCDT). The quantitative relationship between binocular lustre perception and color direction, color difference, and brightness was measured, and the quadratic logarithmic function model of lustre perception and brightness factors was fitted. The experiments and data analysis in this paper shows that brightness has a significant impact on lustre perception, and the research results can provide strong support for lustre reproduction in 3D system design